Mobile sensing system for crop monitoring

ABSTRACT

Described herein are mobile sensing units for capturing raw data corresponding to certain characteristics of plants and their growing environment. Also described are computer devices and related methods for collecting user inputs, generating information relating to the plants and/or growing environment based on the raw data and user inputs, and displaying same.

FIELD OF INVENTION

This disclosure relates to automated systems and methods for monitoring,evaluating, and cultivating crops.

BACKGROUND

Agriculture is the science and art of cultivating plants and livestock.With respect to the production of fruits, vegetables, and other crops,intensive manual labor is required to perform the tedious and oftencostly processes of farming. In addition, successful farming also relieson the experience and manual observation of experienced growers toidentify and improve the quantity and quality of crops produced, forexample, by managing growth cycles, environmental impacts, planting andharvesting timelines, disease prevention, etc. All of which, aretime-consuming and prone to human error.

Accordingly, it would be advantageous to incorporate automated systemsand processes, along with machine learning, to cultivate crops by, forexample, monitoring, treating, and harvesting the crops, along withcommunicating with growers regarding the same.

SUMMARY

Disclosed herein are mobile sensing units for capturing raw datacorresponding to certain characteristics of plants and their growingenvironment. Also disclosed are computer devices and related methods forcollecting user inputs, generating information relating to the plantsand/or growing environment based on the raw data and user inputs, anddisplaying same. The systems and methods described herein are configuredto provide insights to a grower (or other user), such as, for example,environmental and plant conditions relating to at least one oftemperature, plant health status, plant stress status, disease, pests,plant performance, growth status and plant structure, yield predictions,facility mapping, cultivation task records and procedures, diagnosticinformation related to the mobile sensing unit, the facility, and/or theplants (e.g., a proposed course of action relative a plant, such as morewater, apply pesticide, etc.), and temporal and historical data relatedto all of the foregoing.

In one aspect, the disclosure relates to a crop monitoring system. Thesystem includes a mobile sensing unit including a sensor array and acomputing device including a display screen configured to present agraphical user interface. The mobile sensing unit is configured tonavigate arrangements (e.g., rows) of plants within a growing facilityand generate optical data corresponding to the plants. The computingdevice is in communication with the mobile sensing unit. For example,the computing device can be mounted on the mobile sensing unit andhard-wired thereto or can be located remotely and communicatewirelessly. The computing device is configured to at least one ofdisplay the optical data, receive user input related to the opticaldata, generate and display user information about environmental andplant conditions within the growing facility, or receive user input tonavigate the user information about environmental and plant conditionswithin the growing facility.

In another aspect, the disclosure relates to a crop monitoring systemthat includes a mobile sensing unit including a sensor array and acomputing device comprising a processor, at least one memory module, aninput device, and a communication module. In some embodiments, thecomputer device includes a wireless communication module and additionalstorage for providing pre-existing data from one or more databases. Themobile sensing unit is configured to navigate rows of plants within agrowing facility and generate raw data corresponding to the plants. Thecomputing device is in communication with the mobile sensing unit andconfigured to generate user information about environmental and plantconditions within the growing facility. The system further includes agraphical user interface in communication with at least one of thecomputing device or the mobile sensing unit. The graphical userinterface is configured to display the user information aboutenvironmental and plant conditions within the growing facility andreceive user input to navigate the user information about environmentaland plant conditions within the growing facility.

In various embodiments, the graphical user interface is located remotelyfrom the mobile sensing unit, such as outside a growing facility, at amaster control room, or in a free-roaming mobile device. The graphicaluser interface can be used for, for example: navigation and operation ofthe mobile sensing unit, inputting meta-data, which can pertain tocondition of plant at time of data capture or near time of; othermeta-data on plant location, identification, environmental conditions,or task/maintenance conditions (e.g., plants have just been trimmed ormoved or harvested); displaying results and feedback from the systemthat has been generated based on the input data and provide informationon the plants and environments to be used to initiate a course of actionby a user (e.g., either an operator of the mobile sensing unit orcustomer). The computing device can be a laptop or desktop computer, aportable micro-controller (e.g., a tablet, a Raspberry Pi, or aGPU-based computing device, such as NVIDEA Jetson) including a display,where the computing device is programmed to generate, either viasoftware or hardware, graphical images and statistical data relatedthereto.

In various embodiments of the foregoing aspects, the sensor arrayincludes at least one camera with a lens, and optionally a filter, tocapture optical data at different frequency ranges. In some embodiments,the array uses a single camera, such as a FUR Blackfly S area scancamera or a BASLER Racer line scan camera, with multiple,interchangeable lenses and filters, such as, for example, anelectronically-controlled filter wheel and band-pass and long-passoptical filters. The optical data captured can include images acrossvisible and non-visible light spectra, thermal images, and environmentalreadings, such as, for example, at least one of: temperature, humidity,luminosity, radiation, magnetic field, particulate matter, and chemicalcompounds. In various embodiments, the array may include one or morecameras capable of capturing, for example, still images, time-lapsedimages, moving images, thermographic (e.g., infrared) images,one-dimensional images, two-dimensional images, three-dimensionalimages, etc. In another embodiment, the mobile sensing unit is furtherconfigured to emit a light pulse to capture or otherwise sense a plant'sresponse to at least one of visible or invisible light pulses. Thesensor array may also include at least one of a depth sensor, a lightsensor (e.g., a spectrometer), or a thermal sensor that can measure notjust plant features, but also ambient conditions (e.g., facilitytemperature or light intensity/exposure). In a particular embodiment,the sensor array includes a thermal camera configured to generate ahigh-resolution map of temperature variations within the plants. Incertain embodiments, the sensor array includes a swarm system ofautonomous drones equipped with optical sensing devices, where eachautonomous drone is configured to navigate the rows of the plants withinthe growing facility and capture optical data thereof.

In some embodiments, the mobile sensing unit can include a control arm,which can be used to adjust at least one of a height or an orientationangle of the sensor array. The control arm can also be configured tointeract with at least one of the plants or an ambient environment. Forexample, the arm can be used to move or reposition a plant, apply atreatment to the plant (e.g., spraying a pesticide, adding water, etc.),or adjust an environmental aspect (e.g., facility parameter, suchraising or lowering a temperature, initiating irrigation or feeding,applying a treatment, adjusting lighting, etc.). In some cases, thecontrol arm and/or mobile sensing unit can be configured to interactwith one or more actuators (e.g., linear or rotary actuators, such as aNACHI MZ07, or a NACHI SC500, or a WAYLEAD NEMA Motor, etc.), eithermanually or as triggered via a control system, installed within thegrowing facility to adjust at least one of plant spacing, location,orientation, facility parameter, etc. The actuators can be coupled tothe bench, the plant, and/or the growing facility. In some embodiments,the benches (or other plant supporting structures) can be positioned onsliding rails or similar structure, where the actuators can beconfigured to “push” or otherwise move the benches. Additionally, theseactuators can include various control systems, such as lightcontrollers, irrigation/feeding systems, HVAC settings that can beincluded within the various systems described herein or integrated withthe existing facility and equipment controller systems (e.g., hardwareand software), such as Argus Titan II controller, Dosatron irrigationand nutrient delivery systems, Etatron eOne microdoser kit, or HVAC andsoftware greenhouse automation systems, such as those available fromPriva or Argus Controls.

In further embodiments, the mobile sensing unit can be navigatedremotely by a user or navigate the growing facility according to one ormore autonomous navigation algorithms. In some cases, the navigation canbe altered based on data collected from the mobile sensing unit or frominput received from a user via one of the graphical user interfaces thatmay be associated with the computing device.

In still another aspect, the disclosure relates to a crop monitoringsystem, such as those described herein, where the system includesrobotic mechanisms configured to provide plant treatment based on any ofthe data captured or generated by the mobile sensing system, or insightsgenerated thereby. These robotic mechanisms may include, for example,articulated robotic arms, Cartesian robots, drones, autonomous vehicles,humanoids, cable-driven parallel robots, etc. The robotic mechanisms mayprovide treatment based on a system for modelling flower growth to trackthe growth of individual flowers and flower features as describedherein. For example, the model may identify an anomaly in the growth ofthe flowers on a particular plant, and then activate a robot to apply atreatment, reposition the plant, or change an environmental parameter asnecessary. Additionally, the crop monitoring system may be integratedwith one or more climate control systems, plant feeding/irrigation(fertigation), or treatment system within the growing facility and thedata captured or generated by the mobile sensing system, or insightsgenerated thereby, can be used to recommend particular growth parametersor environments and/or activate at least one of the climate controlsystems to modify a condition within the growth facility or a treatmentto a crop, track and certify compliance with applicable regulations, orcombinations thereof.

In additional embodiments, the crop monitoring system may include avision-based system to predict flower or fruit quality (e.g., THCcontent and Terpene profiles, flavor profiles, ripeness, sugar content,content of other organic compounds) and/or an automated system toharvest and sort plants. The automated harvesting system may utilize thevision-based system (e.g., a scanner) and a conveyor system to transport(e.g., via the robotic mechanisms) plants or flowers to differentdestinations based on plant condition or quality. The crop monitoringsystems described herein may include a multi-spectral imaging system todetect unacceptable levels of contaminants in soil, plant, and/orflowers. The systems may rely on a machine learning model configured toidentify the unacceptable levels of contaminants (e.g., biological, suchas microbial, yeast, virus, or inorganic, such as metals) in soil,plant, and/or flowers based on data from a multi-spectral imaging systemand machine learning, which may encompass artificial intelligence anddeep learning concepts, such as, for example, the use of classic neuralnetworks.

The crop monitoring system can present the environmental and plantconditions within the growing facility via one of the graphical userinterfaces disclosed above. This information can include informationrelating to at least one of: temperature, plant health status (e.g.,healthy or sick, growth rate, etc.), plant stress status (e.g., anunfavorable condition or substance that affects a plant's metabolism,reproduction, root development, or growth; such as drought, wind, overirrigation, or root disturbance), disease (e.g., blight, canker, powderymildew), pests (e.g., aphids, beetles, mites), plant performance (e.g.,crop yield), growth status and plant structure (e.g., leaf and canopydensity, branch density, biomass, height, etc.), or yield predictions.In certain embodiments, a user can interact with one of the graphicaluser interfaces in order to display the environmental and plantconditions within the growing facility at a large scale facility-levelor a small scale bench-level. The graphical user interface, inparticular the interface of the second aspect of the disclosure, can beconfigured to generate and present a grid-like map of the growingfacility, including graphical indications of the environmental and plantconditions for individual zones within the growing facility; informationrelated to cultivation task records and procedures; diagnosticinformation related to the mobile sensing unit, the facility, and/or theplants; and temporal and historical representations of the foregoing toidentify trends.

In still further embodiments, the crop monitoring system includes aconveyor system (e.g., a belt and associated drive mechanisms, robots,motorized rails, etc.) configured to transport the plants to a pluralityof locations, such as, for example, another bench within the facility ora location outside or proximate the growing facility, based on datacaptured from the mobile sensing unit. The system also includes a secondsensing unit mounted on the conveyor system and configured to scan theplants traveling on the conveyor system. The second sensing unit can bemounted to the conveyor system using a movable robotic arm or anactuator and configured to capture at least one of two-dimensional orthree-dimensional data at various aspects, such as, for example,lighting, focus, sensor position, or environmental condition. Theconveyor system can be configured to manipulate the plants in order forthe second sensing unit to capture data from various angles.

In yet another aspect, the disclosure relates to a computer-implementedmethod for processing crop information, the method implementing anapplication processing system for use in generating and processinginformation related to environmental and plant conditions within agrowing facility and displaying same. The method includes generating rawdata via a mobile sensing unit, where the raw data corresponds to one ormore characteristics of one or more plants; receiving user inputinformation in one of multiple available input formats through an inputinterface; processing the raw data and user input information to createa curated data set that includes processed images representative of thecrop information; comparing the curated data set against a pre-existingdatabase of domain data; determining, based at least in part on thecomparison of the curated data set, the specific environmental and plantconditions relative to the crop being processed; generating a graphicaluser interface using a GUI generator; and displaying the informationrelated to the environmental and plant conditions.

In various embodiments of the method, the graphical user interface isinteractive and the method further includes manipulating displayedinformation. In addition, a user will have the capability of “markingevents,” which can include an analysis of some of all of the raw datathrough which specific time stamps of interest are deemed to include“significant” or “abnormal” events. Such events can pertain directly tovalues measured by one or multiple sensors.

In still another aspect, the disclosure relates to acomputer-implemented system for presenting information related toenvironmental and plant conditions within a growing facility. The systemincludes a mobile sensing unit including a sensor array and configuredto navigate and arrangement of plants within a growing facility andgenerate raw data corresponding to the plants; an input interface foraccepting user input information in one of multiple available inputformats; application processing components; and a graphical userinterface generator for mediation between the user and applicationprocessing components and displaying same. The computer processorcomponents are programmed to perform the steps of collecting the rawdata and user input information, validating the data and information,automatically selecting one or more decision engines based on the userinput information and a pre-existing database of domain data, selectinga required format corresponding to the selected decision engine from aplurality of available formats stored in a library of decision engineproxies, converting the raw data and user input information intoapplication data according to the corresponding required format, androuting the application data to the one or more selected decisionengines to process the application data; generating information relatedto environmental and plant conditions within the growing facility.

In various embodiments, the computing device includes a display screen,the computing device being configured to display on the screen a menulisting one or more environmental or plant conditions relating to agrowing facility, and additionally being configured to display on thescreen an application summary that can be reached directly from themenu, wherein the application summary displays a limited list of datarelated to or derived from the environmental or plant conditioninformation available within the one or more applications, each of thedata in the list being selectable to launch the respective applicationand enable the selected data to be seen within the respectiveapplication, and wherein the application summary is displayed while theone or more applications are in an un-launched state.

Still other aspects, embodiments, and advantages of these exemplaryaspects and embodiments, are discussed in detail below. Moreover, it isto be understood that both the foregoing information and the followingdetailed description are merely illustrative examples of various aspectsand embodiments, and are intended to provide an overview or frameworkfor understanding the nature and character of the claimed aspects andembodiments. Accordingly, these and other objects, along with advantagesand features of the present disclosure herein disclosed, will becomeapparent through reference to the following description and theaccompanying drawings. Furthermore, it is to be understood that thefeatures of the various embodiments described herein are not mutuallyexclusive and can exist in various combinations and permutations.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to the sameparts throughout the different views. Also, the drawings are notnecessarily to scale, emphasis instead generally being placed uponillustrating the principles of the disclosure and are not intended as adefinition of the limits of the disclosure. For purposes of clarity, notevery component may be labeled in every drawing. In the followingdescription, various embodiments of the present disclosure are describedwith reference to the following drawings, in which:

FIG. 1 is a schematic representation of a growing facility in accordancewith one or more embodiments of the disclosure;

FIG. 2 is a schematic representation of a mobile sensing unit for use inthe facility of FIG. 1 , or similar facilities, in accordance with oneor more embodiments of the disclosure;

FIG. 3 is a schematic representation of a sensor array for use in themobile sensing unit of FIG. 2 in accordance with one or more embodimentsof the disclosure;

FIG. 4 is a pictorial representation of a series of graphical userinterfaces configured for displaying data to a user in accordance withone or more embodiments of the disclosure;

FIGS. 5A-5C are pictorial representations of a process of creating a 3Dpoint cloud reconstruction in accordance with one or more embodiments ofthe disclosure;

FIGS. 6A and 6B are pictorial representations of a 3D point cloudreconstruction of one plant bench in accordance with one or moreembodiments of the disclosure;

FIG. 7 is a graphical representation of the 3D points mapped into the 2Dplane after application of a polynomial degree in accordance with one ormore embodiments of the disclosure;

FIG. 8 is a pictorial representation of two plots showing a scannedsection of a growing facility before and after filtering andsegmentation of the data in accordance with one or more embodiments ofthe disclosure;

FIG. 9 is an enlarged pictorial representation of abutting clustersgenerated via a Kmeans algorithm in accordance with one or moreembodiments of the disclosure; and

FIG. 10 is a pictorial representation of aligned point clouds inaccordance with one or more embodiments of the disclosure.

DETAILED DESCRIPTION

Reference will now be made to the exemplary embodiments illustrated inthe drawings, and specific language will be used here to describe thesame. It will nevertheless be understood that no limitation of the scopeof the disclosure is thereby intended. Alterations and furthermodifications of the inventive features illustrated here, and additionalapplications of the principles of the disclosures as illustrated here,which would occur to one skilled in the relevant art and havingpossession of this disclosure, are to be considered within the scope ofthe disclosure.

FIG. 1 is a bird-eye view of a growing facility 10, which can include,for example, a room or region in an indoor or greenhouse cultivationfacility, a plot of land, or similar (generally denoted with reference12). The systems and methods disclosed herein can be applied tovirtually any size facility having essentially any layout. In someembodiments, the facility 10 (e.g., a greenhouse) includes one or morebenches 14 or other structures (e.g., shelving, crop rows, hangingstructures, etc.) that hold, house, or otherwise support the plants in aparticular orientation. Generally, the benches/plants will be laid outin a series of rows having access paths 16 located therebetween. Whilethe plants are typically described as laid out in rows, they can also besupported on vertical growing structures, such as tiered benches orvertical farming systems, with the sensing unit 20 navigatingaccordingly (e.g., moving vertically, horizontally, or diagonally asneeded). Typically, the benches and plants are spaced some distanceapart along each row and/or vertical tier, which may be determined basedon the type of plant, its stage of development, etc. As shown in FIG. 1, there are a series of benches 14, each with one or more plantsdisposed thereon in different stages of development.

Also shown in FIG. 1 is a mobile sensing unit 20, which is configured tomove throughout the facility 10, either autonomously, via humaninteraction, or both, collecting data from and interacting with theplants, and in some cases, the benches 14, as well. The mobile sensingunit 20 will be described in greater detail with respect to FIGS. 2 and3 . Generally, the mobile sensing unit 20 moves up and down the rows 16,stopping at each bench or specific benches, collecting data from theplant and/or its surroundings, manipulating the plant or bench asnecessary, and returning to a home position.

As described in greater detail below, the path of the mobile sensingunit 20 can be navigated via human input and/or autonomously and may bealtered based on the data previously collected or certain “triggers”within the facility, such as, optical tags, magnetic tags, colorstripes, etc. The mobile sensing unit 20 may also include a positioningsystem.

An exemplary version of the mobile sensing unit 20 is depicted in FIG. 2. Generally, the unit 20 includes a base 26 that includes a locomotionsystem 28, such as a set of wheels or tracks coupled to, for example, amotor and/or cables. Other means for propulsion, which would be known toa person of ordinary skill in the art, are contemplated and consideredwithin the scope of the disclosure. The locomotion system 28 can bemechanically or electronically controlled, either by a controller systemoperated by a human or as controlled by an autonomous navigationalgorithm. The locomotion system 28 and propulsion system will beconfigured to suit a particular application (e.g., size of the growingfacility, terrain, environmental conditions, or environmentalregulations). The unit 20 also includes a sensing package or array 22that can include at least one sensing camera, or other data capturingdevice, and may include depth sensing and other types of optical orenvironmental sensing devices, as described below. The sensing package22 can be disposed atop structure 24 extending upwardly from the base26.

The following is a list of exemplary sensor devices that can beincorporated in to the sensing system 20: CMOS sensors, such as aBlackfly S USB3 (MODEL: BFS-U3-200S6C-C: MP, 18 FPS, SONY IMX183) (seehttps://www.flir.com/products/blackfly-s-usb3/?mode1=BFS-U3-20056C-C),Blackfly S Board Level (MODEL: BFS-U3-200S6C-BD2: 20 MP, 17 FPS, SONYIMX183) (seehttps://www.flir.ca/products/blackfly-s-board-level/?model=BFS-U3-200S6C-BD2);Calibrated Thermal Cameras, such as calibrated and uncalibratedradiometric imagers (seehttps://infraredcameras.com/thermal-infrared-products/8640-p-series/);line scan cameras and hyperspectral line scan cameras, such as BASLERRacer line scan cameras (seehttps://www.baslerweb.com/en/products/cameras/line-scan-cameras/);hyperspectral line scanning cameras (seehttps://www.ximea.com/en/products/xilab-application-specific-oem-custom/hyperspectral-cameras-based-on-usb3-xispec?gclid=CjwKCAiAo7HwBRBKEiwAvC_Q8S7UDCLrZmWn8LZ8ZW61g3GFyyHGF-t2eAPZyd3vSwzr_VbV4UGw4BoCH4IQAvD_BwE);Short-wave infrared cameras, such as SWIR HDR Gated InGaAs camera (seehttps://axiomoptics.com/11c/widy-swir-ingaas-cameral); optical lensfilters, such as band-pass filters (seehttps://midopt.com/filters/bandpass/) or long-pass filters (seehttps://midopt.com/filters/longpass/); Machine Vision Camera lenses,such as Azure fixed and vari-focal length lenses seehttps://www.rmaelectronics.com/azure-lenses/); Multi-spectral camerasystems with optical filter wheels (CMOS-based sensors), such as customor off-the-shelf camera-lens-filter systems (seehttps://www.oceaninsight.com/products/imaging/multispectral/spectrocam/);Multi-spectral cameras using dichroic filter arrays or sensor arraystuned to specific spectral bands seehttps://www.oceaninsight.com/products/imaging/multispectral/pixelcam/,https://www.oceaninsight.com/products/imaging/multi-band-sensor/pixelsensor/per-fluorescence-sensor/);Spectrometers and light and PAR sensors (seehttps://www.oceaninsight.com/products/spectrometers/,https://www.oceaninsight.com/products/systems/,https://www.apogeeinstruments.com/quantum/); Environmental sensors forhumidity, temperature and gases (CO2, volatile organic compounds) (seehttps://www.apogeeinstruments .com/humidity-probes/,https://www.vaisala.com/en/products?mid=%5B876%5D). Additional datecapture sensors/devices include stereo camera systems and/or LIDAR(light detection and ranging) systems for three-dimensional (3D)mapping.

In some embodiments, the structure 24 includes means (e.g., a verticaltelescopic or scissor lift) for manual or automatic height adjustment ofthe sensing package 22 (the height adjustment can be manual orelectronic by means of an actuator, such as a linear actuator driven byan electric motor or pneumatic pressure) to, for example, capture theoptical data of plants of differing size, different locations, or fromdifferent perspectives. In an alternative embodiment, the heightadjustment is done via a drone system that can launch from and land onthe base 26 vertically using, for example, propellers, lighter than airaircraft design, or electro-magnetic force. The navigation system can beshared between the drone and the mobile sensing unit 20. Any of thesestructures 24 (i.e., actuators or drones) can be manually controlled bya human or electronically controlled with a control unit operated by ahuman or autonomously using a control algorithm. In addition, theorientation and/or operation (e.g., height, angle, focus, lighting,capture rate, etc.) of any of the cameras or other image/data capturedevices can also be controlled mechanically or electronically asdescribed above. In various embodiments, the structure 24 may include arobotic arm 25 extendible therefrom that can be deployed to interactwith the plants, benches, or other structures within or part of thefacility 10, as known to a person of skill in the art.

Typically, the unit 20 will also include a computing device, which canbe located within the base 26 or sensor package 22 with batteries or acable to connect to a power source. The computing device is configuredto capture/store data (e.g., raw data captured by the sensors, datainput by a user, or a pre-existing database of domain data) and/ortransmit the data to the cloud. This or another computing device may beused for controlling and handling communication between any electronicelement in the unit, such as, for example, actuators, cameras(triggering other sensors, communicating with sensors or controllerunits, etc.). The computing device(s) can be connected to sensors viawires or remotely through wireless communication technologies. In someembodiments, the computing device can be housed elsewhere within themobile sensing unit 20 or be housed remotely, for example, as acompletely separate unit when all communications are wireless. Thecomputing device may or may not perform some preprocessing on the datathat is being collected, referred to herein as edge computing.

FIG. 3 depicts one embodiment of the sensor package 22 and certaincomponents thereof. A top view of a set of cameras 30, each with aspecific lens 32 and a spectral band filter 34; however, in someembodiments, only a single camera can be used in combination with arotating spectral filter, and/or other sensors as disclosed herein tocapture data. Generally, the cameras 30 capture visual features in thevisible and invisible spectrum from each plant, such as single ormultiple images per plant or a video recording of the plants and/or ascan of the crops and/or growing facility. Each camera unit includes alens 32 and optionally a filter 34 that can be mounted on the outside orinside of the lens or between the camera and the lens 32. Thecamera/sensor 30 is connected to a computing device, such as one that ispart of the mobile sensing unit 20 or one located remotely (e.g.,somewhere in the greenhouse), either wirelessly or via a UniversalSerial Bus (USB) connector or Ethernet port.

In an alternative embodiment, a filter wheel that contains filters ofdifferent spectral bands is integrated with a camera to allow the camerato capture optical data of different frequency ranges using the sameoptical sensor. An electromechanical system can be used to change thefilter position to bring the appropriate filter in front of the camera.Alternatively, the filter 34 may be composed of a material that changesfrequency ranges in response to either mechanical pressure or anelectromagnetic field applied to it. In some embodiments, the mobilesensing unit 20 includes a flashing system that emits a pulse at oraround the time of data capture to sense how a plant responds to avisible or invisible light pulse, referred to herein as active sensing.The flashing system may be part of the sensing package 22 or separatefrom it.

The mobile sensing unit 20 uses a control system operated by a human oran algorithm or computer software triggered by a human to capture datausing the cameras (and/or other sensors) synchronously orasynchronously. Generally, control algorithms use the sensor datacollected from visual information in the field of view of the sensors toadjust some of the parameters associated with the mobile sensing unit 20to optimize the quality of the data collected and/or to navigate themobile sensing unit, where, for example, the unit 20 operates semi- orfully autonomously. This data is then transferred manually or via cloudto another computing device for further pre-processing andpost-processing.

In some embodiments, the sensor array is configured to provide data forVapor-Pressure Deficit (VPD) mapping. Generally, the array includes oneor more thermal cameras that sense short-wave, mid-wave or long-waveinfrared spectrum to generate a high-resolution map of temperaturevariations in the canopy. This data combined with other environmentaldata can produce an approximation to a high-resolution map of VPD, whichis a measure of a plant micro-environment. VPD mapping can be used tohighlight and detect regions or “hot-spots” with increased risk forplants to suffer from an undesirable condition, such as powdery mildew.An end-user then can be notified when such conditions are detected, orthe system can trigger certain actions manually or autonomously viahumans or machines, such as adjusting an environmental variable atplant/bench/room level or directing another device to the affected areato apply a treatment. The aforementioned sensors can also be used todetect particular features of the plants themselves, such as, forexample, water stress or signs of pathogens.

In various embodiments, the sensing package 22 may be replaced with aswarm system of small drones, lighter-than-air-aircrafts, or biomimicryflying robots (e.g., having wings instead of propellers). These dronescan be equipped with small optical sensing systems that capture opticaldata at some resolution and at post-processing converts this data into amore enriched version of the entire canopy at plant level or at someother level of granularity. The swarm of robotic flies/drones take offand land from a base that can be similar to the one discussed above withrespect to FIG. 2 . The drones may be navigated by a central algorithmthat navigates them collectively or they can be navigated autonomouslyby individual/independent control units deployed on a computing deviceattached to or in communication with the drone. The data captured inreal-time from a set of sensors or optical sensors can be fed to thiscontrol algorithm to optimize the navigation so as to make sure thereare no collisions and to make sure the scan is completed within adesired time and covers all of the areas of interest.

In additional embodiments, the sensing system 20 is configured toperform various data capture steps that provide for plant-levellocalization for providing plant-level stress mapping and analyticsinsights to growers to improve cultivation by loss prevention throughoptimal and targeted treatments. Performing localization processes mayfurther enhance the data and insights by reducing or eliminating errors(e.g., false positives), improving resolution and focus for providinginsights to the grower. As previously described, the sensing system 20scans rows of plants and automatically or manually through the use ofhuman input via software assigns certain location information. This canbe done completely manually to completely automated or using a hybridapproach through a combination of techniques including, but not limitedto, QR code detection, wheel and visual odometry. In some cases, thisstep of segmenting the data and assigning it to room/row or moregranular regions levels may not be enough to do a 1:1 mapping betweenraw or processed collected data to individual plants, in which case, 3Dmapping can improve this process. Additionally, during data/imagecollection and/or data/image processing, the system may assign asignature or “fingerprint” to each plant that remains assigned to theplant throughout the various processes and growth phases, such that thesystem can readily identify a particular plant and/or its location atessentially anytime.

In some embodiments, creating a 3D map of individual plants or pots ofplants allows the system/user to detect them in the 3D space. Generally,the term pot is used to designate distinct containers or other uniquestructures (e.g., hydroponic pods, grow trays, shelves, troughs, etc.)that can be correlated to a particular plant or set of plants. The mapcan be created using a 3D mapping sensor such as a stereo camera system,LiDAR, or other technologies capable of generating such maps. The fusedcloud point of each region of plants can then be segmented before orafter preprocessing to correct visual odometry in order to create acluster of points referring to each plant or pot corresponding to eachplant. Next this data is projected into the point-of-view (PoV) from theinferred position of the camera(s) used during the sensing (e.g., aspart of 3D scanning or as separate RGB, spectral, or thermal cameras).The projected clusters can then be used as masks for the 2D imagescollected during the data collection process to provide a 1:1relationship between individual plants and a subset of data availablefor each plant in the 2D world.

Other embodiments may use a combination of pot detection as well asinferred plant height profile to generate a simulated model of theindividual plant profile before projecting into the 2D point of view ofeach camera at the time of capture for each capture point. Plant levelinferred height profile can be a useful metric to detect growth relatedcharacteristics (such as, for example, size, leaf density, growth rate,other nominal features, and anomalies) by itself and can be provided tosystem users as a 2D or 3D map to high-light regions of interest fortreatment, predict growth, and/or to categorize pace of growth forvarious type of actuations to improve the cultivation operation.

In some embodiments, plant localization approaches include using an IRcamera and an RGB camera to collect 2D Images of the plants. A plantmask is created by extracting the pixels associated with plants by, forexample, thresholding the pixel values in the image. Specific types ofplants may be found using clustering algorithms, such as Kmeans orHoperaft-Karp. The same plants may be mapped between images usingoptical flow methods and graphical methods; however, this method has itslimitations. For example, images taken of plant canopies are verydifficult to segment, even with the human eye. A major reason behindthese issues is that the perspective change between images causes thesame region of the image to look completely different, resulting inplant segmentations that are not very accurate, often cutting plants inhalf.

This process may involve creating and fusing two separate point cloudsto create a holistic 3D plant and pot profile for the localizationpurposes and lab calibration techniques used to optimize the fusionparameters and transformation between various camera frames in the 2Dand the 3D worlds. Additionally, the depth information can also beoverlaid with the 2D pixel values such as spectral RGB and thermal tocreate an enriched set of data for plant level analytics beyond what asingle or set of 2D individual plant data can offer through machinelearning techniques, such as various architectures available for 3Dconvolutional networks. The process may also use QR/April tags, andthrough real-time detection of those tags, assign the right meta-dataabout the location where the images where taken and theplants/regions/benches/trays they correspond to. The tags can bedetected in the images to help with localization as well as improvingthe 3D point cloud fusion and addressing noises and artifacts that mayarise due to errors in visual odometry.

The data captured and mapped as disclosed above can be used to provideinsights to a grower. An auto-scanner records hundreds of gigabytes ofdata of the plants, etc.; however, the processing of this data is laborintensive if done by hand. Accordingly, as much as possible, the dataprocessing side of providing the data to insight pipeline should beautomated, especially, the mapping of plant data.

In some cases, the auto-scanner records plant data based on a timer andwithout a rigorous mapping between the data recorded and which plantthat data is associated with. This means that the insights that theauto-scanner is able to provide has limited precision, specifically forproviding insights on a specific plant. While the auto-scanner is ableto tell a worker if the plants have an issue, it is not able to tellthem which plant. The objective of the data to insights (D2I) pipelineis to connect the raw data generated by the auto-scanner and process itto make plant level insights more accessible. In order to do this theD2I pipeline must include some sort of plant localization, as discussedherein, where plant locations are extracted from raw data generated bythe auto-scanner.

In a particular embodiment, the system extends the data from 2D to 3D byusing point cloud data, as disclosed above, which allows the system totake advantage of 3D reconstruction algorithms that give data that isrelatively consistent across different fields of view. In some cases,this approach includes collecting images of pots (or other containers,etc.) rather than canopy for localization, which allows the system tobetter estimate plant locations, because the positions are much clearer.Another added advantage is that the system can concatenate the 3D pointclouds into a larger bench wide point cloud, allowing the system toanalyze the entire bench in one dataset. To further augment thecapabilities, the 3D scanning may be done with two cameras. One camerapointing to the canopy and the second camera pointing to the pots, whichalso allows the system to get a prediction of plant height and also usethe pot locations for plant localization. Generally, the processincludes creating 3D point cloud reconstruction, mapping point cloud toa world frame, removing distortions introduced by simultaneouslocalization and mapping (SLAM), extracting pot positions, combiningcanopy points clouds, and extending solution to two cameras, asdescribed below. In some embodiments, the system uses a depth camera(e.g., the D435i RGBD camera as available from Intel® in Santa Clara,Calif.) with an onboard inertial measurement unit (IMU) pointed at theplant pots.

To create the 3D point cloud reconstruction, the SLAM algorithm is usedand relies on the

IMU and visual odometry from the camera. The SLAM algorithm uses RoboticOperating systems (ROS) rtabmap library and outputs a point cloud data(PCD) file, which saves the data as a colored point cloud. One exampleof a 3D point cloud for a bench is shown athttps://share.getcloudapp.com.

Mapping the Point Cloud to a World Frame is carried out in a pluralityof steps as follows (see FIGS. 5A-5C). The PCD file is the read usingthe open3D Python library. The coordinate system of the point cloud 210has its origin centered at the camera and the axes oriented along thecamera look at vector. To better extract insights, the system projectsthe points into the world frame 200. The coordinate system has the XYplane 202 lying on the table plane, although other systems arecontemplated and considered within the scope of the disclosure. Mappingthe XY plane onto the table plane includes rotating the axes, globallyaligning the measured parameters, estimating the table plane using theleast squares, and the local alignment based on a normal vector of thetable plane.

The coordinate axis is rotated so that the X axis 202 b points along thebench, the Y axis 202 a is the camera view at vector and the Z axis 206points up relative to the camera. Using the camera angle (Beta) andrelative height from the camera to the table, the system rotates andtranslates the coordinate axis accordingly. Global alignment results inthe Y axis 202 a pointing towards the plants and as parallel to thetable plane as possible, with the Z axis 206 pointing up. The XY plane202 should be as close to the table plane as possible. The table planeis estimated by filtering the point cloud based on the Z coordinate, andkeeping points where the absolute value of Z is within some designatedor otherwise relevant threshold. The least squares are then used to fitthe points to a plane. FIG. 5A also depicts the estimated mesh plane204. This is before local alignment; thus the coordinate axis is offsetfrom the table plane. FIG. 5B depicts a different view of a similarpicture where only the thresholded points are shown and the table planemesh appears to fit the flat surface created by the points clouds. Localalignment is carried out by calculating a rotation matrix based on anormal vector of the plant. For example, rotate the table plane mesh andthen find the Z offset to get a translation vector. With the rotationmatrix and translation vector, the system can fine tune the point cloudpositions. See FIG. 5C.

As shown in FIG. 5A, the point cloud is plotted with the table planemesh 204 (shown in purple). The green arrow 202 a (Y axis) is notaligned with the table plane and an offset is illustrated. FIG. 5Bdepicts an accurate estimation of the table plane despite errors inglobal alignment, with the thresholded point clouds 210 in brown and thetable plane mesh 208 in yellow. After the local alignment step, themisalignment between the coordinate system and the table plane mesh 208is removed after local alignment. The green arrow should be aligned withthe purple mesh 204 and the table in the 3D point cloud as shown in FIG.5C.

The 3D reconstructed scene 300 is generated using a SLAM algorithm thatcombines the camera IMU and visual odometry. However, errors in poseestimation can build up over time to cause estimated pose to drift fromthe true pose. This drift 350 is shown in FIG. 6A, which depicts the 3Dreconstruction of one plant bench, which is a straight line in reallife, but curves across the X-axis. The system is configured to removethe distortions introduced by the SLAM algorithm. The method includesmodeling the curved profile of the point clouds as a polynomial curvethat is a function of x and finding the transformation that will mapthese points to a line.

The method includes mapping 3D points to the 2D so that they now sit onthe XY plane 600. The Z coordinates of the data are considered to beaccurate and can be ignored because of the local alignment step utilizedin mapping the point cloud to the world frame. After finding the bestfit line for the data, the data is transformed. Obtain the parameters mand b from y=mx+b. Then translate the point cloud so that the best fitline aligns with the x axis. Use a least squares method to find the bestfit polynomial to the data. In the example illustrated in FIG. 7 , apolynomial degree of 3 was used and shows the sampled 2D points 610 andthe fitted polynomial to the curve 625. The points considered forpolynomial fitting were randomly sampled with a sample size of 100. Thepolynomial found p(x) returns y for a value of x.

Next, the points are translated according to the polynomial function.Equation: Y_f=Y_0+f (X_0), where the final point cloud coordinates are[X_f, Y_f, Z_f] and the initial coordinates are [X_0, Y_0, Z_0] andZ_f=Z_0 and X_f=X_0. FIG. 6B depicts the 3D transformed point cloud.After the correction, the points clouds are moved so that they arecentered along the x axis, which mostly removes the curved distortions.While the larger distortion is removed, there is still an artifactpresent and the bench is not completely straight. Changing the samplingmethod/number of samples and/or the polynomial degree that areconsidered in the fitting of the dataset to a polynomial should improvethe result. After this process is carried out, the Z coordinate of eachpoint is preserved. As such, projecting to 3D can be done simply byadding the original Z coordinate to each associated 2D point.

After carrying out the steps described above, the pot positions arerelatively easy to extract. To extract the pot positions, the systemfilters the points so that only the points that are within a certainthreshold of the pot rim height are kept. These points can be projectedonto the 2D, and then further clustering and filtering is done toextract the pot positions. Specifically, the pot heights can be used asfilter points, because the system knows the exact height of the pots itfilters points by their z axis values, only keeping points that arewithin a certain threshold of the pot rim height. The filtered pointsare projected onto an occupancy grid. For example, the 3D points aremapped to 2D and the system creates a 2D occupancy grid, scaled by thevoxel size used to down-sample the 3D points. The 2D points are mappedto a cell in the occupancy grid, where each item in the occupancy gridis either set to 1 or 0 depending on if a 2D point is mapped to it ornot.

Next, a clustering algorithm (e.g., the Hoperoft-Karp ClusteringAlgorithm) is used to generate a list of clusters where cells in theoccupancy grid that share an edge are assigned to the same cluster. Atypical pot has a certain dimension and when mapped to the occupancygrid, that dimension should correspond to some area value (e.g., rightsizes). If it is within some minimum and maximum threshold, it isaccepted as a pot. The coordinates of the centroid are chosen as the potpositions. However, if the cluster area is too small compared to atypical pot area, it is rejected. If it is too large, then it is passedfor further processing.

In some cases, large clusters could actually be multiple set of potsthat just happen to belong to the same cluster when it was projectedinto 2D. This is likely because the pots were too close to begin with.In order to separate these pots, the system estimates the number ofplants using the ratio K, where K is equal to(total_cluster_area)/(typical_pot_area). This is the value of K that ispassed into a Kmeans algorithm for segmentation. The Kmeans processshould divide up the overlapping clusters into K separate clusters. Thecentroids of these new clusters are then returned as plant centers.Large cluster processing benefits from tuning of the thresholds fromfinding the right size clusters and the estimation of the typical potsize.

The result of this process is shown in FIG. 9 , while FIG. 8 depicts theoccupancy grid after application of the clustering algorithm (top plot)and the Kmeans algorithm (bottom plot). Specifically, the top plot ofFIG. 8 depicts the clustered occupancy grid and includes a lot of smallclusters that are not pots. The bottom plot of FIG. 8 depicts theclustered occupancy grid after filtering and segmentation based on area,where only cells that confidently correspond to pots are colored. Thedifferent colors are used to distinguish between different pots. FIG. 9is a close-up of a cluster that is touching (technically two abuttingclusters) and was originally recognized as only a single cluster afterusing the Hoperaft-Karp algorithm. The Kmeans algorithm is able tosegment the abutting clusters into two separate clusters.

The images captured, generated or otherwise derived from the capturedimages may be further enhanced by, for example, using two cameras (e.g.,both on the sensing unit 20, one camera located on the sensing unit 20and a second camera or cameras located throughout the facility, or anynumber of cameras on individual drones). In order to infer plant heightfrom the 3D reconstruction, the system can use data from the point cloudof the canopy and integrate these two sets of point clouds. In somecases, this also results in a curved profile that might not necessarilymatch the profile of the pot scene, making it difficult to directlytransform the points into the correct position. In some embodiments, anew point cloud topic is created in ROS that has the integrated pointclouds from the pot camera point of view. The SLAM mapper is used to mapthis point cloud. The method takes segments of the canopy point cloudand uses the iterative closest point (ICP) or random sample consensus(RANSAC) algorithm to match them in the right place; however, incorrectmatching may occur. This may be improved by overlap between the twopoints clouds, with greater overlap resulting in fewer errors. In thismethod, the ROS code uses the rtab_map library and the launch file isbased off the demo_two_kinect.launch: link.

The transformation between the two cameras must be accurate in order tocombine the two sets of point clouds. Manually measuring the transformbetween the two cameras is both cumbersome and prone to errors, becausethe positions of the cameras have to be adjusted often to accommodatedifferent data capturing scenarios. Accordingly, measuring the transformevery time is very labor intensive and undesirable and, therefore, acomputational approach is used. The computational approach uses theopen3D registration library for RANSAC and ICP to find thetransformation between two sets of point clouds. The result of runningthis algorithm is shown in FIG. 10 . As shown, the two previouslyunaligned point clouds 570, 560 (shown in red and teal) are aligned.This computed transform needs to be converted into the ROS coordinatesystem. A library called pyrealsense, which was used to save the pointclouds for calibration, saves the point clouds using a differentcoordinate system than the one dual camera ROS program uses for the 3DReconstruction. In FIG. 10 , the red, green, and blue arrows correspondto the x- y- and z-axes.

The navigation strategy may be dictated autonomously or via human input.The navigation may be altered based on the data collected previouslyusing the same sensors or using a different sensing system. Thenavigation system/strategy can also utilize an indoor positioning systemthat may be used in addition to other methods to associate each piece ofrecorded data with a specific location or plant. Additionally, opticalprinted tags or RFID tags may be used on each plant to associate theoptical data with a certain plant, or a location within the facility.Magnetic or color stripes can also be used (e.g., attached to the groundor other structure within the facility) to help the navigation systemguide the unit 20.

In various embodiments, the growing facility may include variousmechanisms (e.g., an actuator) that the mobile sensing unit 20 caninteract with to adjust some characteristic or other variable feature ofthe growing facility. For example, the mobile sensing unit 20 couldinteract with (e.g., via a robotic arm 25 or a wireless control signal)one or more actuators or drivers coupled to the benches or doors toadjust a spacing between aisles to allow the unit 20 to enter a specificarea or to allow the unit 20 to enter or exit a certain room. In otherexamples, the unit 20 could adjust an environmental setting within thefacility or a specific area thereof, such as increasing or decreasingtemperature, humidity, or lighting levels.

FIG. 4 depicts a series of graphical user interfaces (GUI) 400 that canbe configured to deliver data and insight to an end user (e.g., acustomer, a cultivator, or operator of a cultivation facility).Generally, the data and insight outputs are generated by the systemsdescribed herein, with, in certain embodiments, essentiallyinstantaneous delivery of insights while collecting the data. Theseoutputs are then delivered via a custom-designed software userinterface, an example of which as shown in FIG. 4 . The custom softwarecan be web-based, hosted on a cloud-based software system, and connectedto a pre-existing database of domain data, model outputs,recommendations, etc. that are part of the system. In some embodiments,the GUI 400 can be part of the mobile sensing unit 20, can be locatedremotely, or both. For example, the GUI 400 can be mounted outside eachgrowing area (a room or greenhouse section) or be incorporated into afree-roaming mobile devices, such as tablet devices and smart phonedevices. Furthermore, the GUI 400 can be interactive allowing an enduser to cycle through different sets of data, run diagnostics, updatethe insights, or just input data generally.

The GUI can be accessed in two primary functional forms. The softwareinterface can be run on a tablet device(s) 400A, 400B, 400C, which canbe mounted outside a growing facility (410 in FIG. 4 ). The GUI 400presents a series of screens (i.e., pages) that provide cultivatorsworking in the facility access to information about the environmentaland plant conditions inside the growing facility (e.g., temperature,plant health status including presence of stress, disease or pests,plant performance (e.g., growth status, yield predictions). Theseconditions can be the present conditions, historical conditions, orcomparisons thereof (e.g., a graphical representation of a growth cycleof a plant and the environmental conditions during the cycle). The GUIcan present these insights and data at different scales, including anoverview of the room or growing area and more detailed “bench-level”(including plant-level) information presented in a grid or matrix thatmimics the layout of the facility and crop. This allows the cultivatorsto understand the status of the crops, while minimizing human exposureto the crops. It also allows the end user to track and managecultivation tasks via targeted and time-optimized methods, rather thanblanketed treatments and ad-hoc timing.

Alternatively or additionally, the GUI can be accessed through desktopor laptop computer(s) (400D) to provide the same information asdescribed above, but can also include additional data representationsand time-series trend analysis that visualizes crop performance (health,yield, instances of stress, instances of environmental issues thataffect plant growth) and can be filtered and viewed based on metadatafields (strain, crop cycle or number, room/facility area) and includescultivation task records that are also visualized based on time-series,man-hours or crop cycles (e.g., plant de-leafing tasks performed,integrated pest management tasks scheduled and performed, water andnutrient treatments, soil amendments etc.). This data is used by the endusers to analyze cultivation procedures and practices, optimize humanresources and minimize exposure, and perform proactive supply planning.

Additional aspects of the systems described herein can be coupled with aconveyor system 40 (see FIG. 1 ) that can be configured to handle theplants or benches to, for example, reorient or relocate them. In somecases, the plants may be moved to allow for additional data collection.In some embodiments, the conveyor system 40 includes one or moreconveyor belts (or other transport mechanism, such as an overhead crane,cable-driven parallel robots, or sliding benches) coupled with anynecessary drive systems. The conveyor system 40 can be incorporated witha sensing system 42, such as the mobile sensing unit 20 or a separatesensing unit 42 disposed on the conveyor system 40. In one aspect, aconveyor system 40 and sensing system 42 is configured for pre-harvestor post-harvest predictive grading, issue/anomaly detection, etc.

In certain embodiments, a sensing unit 42 mounted on a conveyor beltscans plants that are transported by or otherwise presented to thesensing unit at a certain pace. The plants may be introduced to theconveyor belt by actuators incorporated within the conveyor systemdesign. The conveyor belt can route different plants to differentlocations based on a decision made by a human after seeing insightsacquired through the crop-scanning software through an interface (suchas the GUIs described above) or an algorithm can autonomously navigatethe plants to different locations based on the insights it gets from theresults of the scans (e.g., the data captured via the sensing unit andprocessed for quality grading predictions or stress/anomaly/diseasedetection).

In various embodiments, the sensing unit 42 can be attached to a roboticarm or an actuator that allows the sensing unit to capturetwo-dimensional and three-dimensional data from the entire 360-degreefield of view. In some embodiments, the conveyor belt may be designed torotate the plants for this to happen. The conveyor belt can also existto navigate post-harvest material and a similar scanning system can bemounted to collect scans on the belt throughput. Again, an algorithm maybe used to actuate different robotic parts in the conveyor system orseparate robotic arms to route material to different locations or toapply certain agents or environmental conditions to different plants orareas.

The data captured (and processed) by the sensing unit can be associatedwith the post-harvest data collected at various stages of materialprocessing. This data can then be used for supervised or unsupervisedtraining of statistical/machine learning models for qualitygrading/scoring. Additionally, the data collected by the sensing unitfrom all the post-harvest plant material, which will be processed forextraction together at a later time, can be used for inference andprediction of yield quality and volume, can be used to modify the recipeof how the material will be processed in the following steps in theentire process of delivering it to an end-user or a customer, or informany decisions made throughout that process such as pricing, etc.

Having now described some illustrative embodiments of the disclosure, itshould be apparent to those skilled in the art that the foregoing ismerely illustrative and not limiting, having been presented by way ofexample only. Numerous modifications and other embodiments are withinthe scope of one of ordinary skill in the art and are contemplated asfalling within the scope of the disclosure. In particular, although manyof the examples presented herein involve specific combinations of methodacts or system elements, it should be understood that those acts andthose elements may be combined in other ways to accomplish the sameobjectives.

Furthermore, those skilled in the art should appreciate that theparameters and configurations described herein are exemplary and thatactual parameters and/or configurations will depend on the specificapplication in which the systems and techniques of the disclosure areused. Those skilled in the art should also recognize or be able toascertain, using no more than routine experimentation, equivalents tothe specific embodiments of the disclosure. It is, therefore, to beunderstood that the embodiments described herein are presented by way ofexample only and that, within the scope of any appended claims andequivalents thereto; the disclosure may be practiced other than asspecifically described.

The phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. As used herein, theterm “plurality” refers to two or more items or components. The terms“comprising,” “including,” “carrying,” “having,” “containing,” and“involving,” whether in the written description or the claims and thelike, are open-ended terms, i.e., to mean “including but not limitedto.” Thus, the use of such terms is meant to encompass the items listedthereafter, and equivalents thereof, as well as additional items. Onlythe transitional phrases “consisting of” and “consisting essentiallyof,” are closed or semi-closed transitional phrases, respectively, withrespect to any claims. Use of ordinal terms such as “first,” “second,”“third,” and the like in the claims to modify a claim element does notby itself connote any priority, precedence, or order of one claimelement over another or the temporal order in which acts of a method areperformed, but are used merely as labels to distinguish one claimelement having a certain name from another element having a same name(but for use of the ordinal term) to distinguish claim elements.

What is claimed is:
 1. A crop monitoring system comprising: a mobilesensing unit including a sensor array and configured to navigate anarrangement of plants within a growing facility and generate opticaldata corresponding to the plants; and a computing device including adisplay screen configured to present a graphical user interface, whereinthe computing device is in communication with the mobile sensing unitand is configured to at least one of: display the optical data; receiveuser input related to the optical data; generate and display userinformation about environmental and plant conditions within the growingfacility; or receive user input to navigate the user information aboutenvironmental and plant conditions within the growing facility.
 2. Thecrop monitoring system of claim 1, wherein the sensor array comprises atleast one camera with a lens and filter to capture optical data atdifferent frequency ranges, and at least one of a depth sensor, a lightsensor, a thermal sensor, a thermal camera configured to generate ahigh-resolution map of temperature variations within the plants, orcombinations thereof.
 3. (canceled)
 4. The crop monitoring system ofclaim 1, wherein the mobile sensing unit further comprises a sensorarray control arm configured to adjust at least one of a height or anorientation angle of the sensor array. 5-6. (canceled)
 7. The cropmonitoring system of claim 1, wherein the mobile sensing unit navigatesthe growing facility according to one or more autonomous navigationalgorithms.
 8. The crop monitoring system of claim 1, wherein the mobilesensing unit is configured to interact with actuators installed withinthe growing facility to adjust at least one of plant spacing, location,orientation, lighting, irrigation, feeding, HVAC settings, or otherfacility parameter.
 9. The crop monitoring system of claim 1, whereinthe sensor array includes a swarm system of autonomous drones equippedwith optical sensing devices, wherein each autonomous drone isconfigured to navigate the rows of the plants within the growingfacility and capture optical data thereof.
 10. The crop monitoringsystem of claim 1, wherein the environmental and plant conditions withinthe growing facility are presented via the graphical user interface andinclude information relating to at least one of: temperature, planthealth status, plant stress status, disease, pests, plant performance,growth status, light intensity, humidity levels, or yield predictions.11. The crop monitoring system of claim 1, wherein the system includesrobotic mechanisms configured to provide plant treatment based on any ofthe data captured or generated by the mobile sensing system, or insightsgenerated thereby.
 12. The crop monitoring system of claim 11, whereinthe robotic mechanisms provide treatment based on a system for modellingflower growth to track the growth of individual flowers and flowerfeatures.
 13. The crop monitoring system of claim 1, wherein the cropmonitoring system is integrated with one or more climate controlsystems, plant feeding/irrigation or treatment systems within thegrowing facility and the data captured or generated by the mobilesensing system, or insights generated thereby, can be used to recommendparticular growth parameters or environments and/or activate at leastone of the climate control systems to modify a condition within thegrowth facility or a treatment to a crop, track and certify compliancewith applicable regulations, or combinations thereof.
 14. The cropmonitoring system of claim 1, further comprising: a conveyor systemconfigured to transport the plants to a plurality of locations based ondata captured from the mobile sensing unit; a second sensing unitmounted on the conveyor system and configured to scan the plantstraveling on the conveyor system, and one or more of: a vision-basedsystem to predict the quality of individual flowers and flower features,fruits, vegetables, or other plant structures; a multi-spectral imagingsystem to detect unacceptable levels of contaminants in soil, plants,flowers, or combinations thereof; an automated system to harvest andsort plants, wherein the automated harvesting system utilizes thevision-based system and conveyor system to transport plants or flowersto different destinations based on plant condition or quality; thesecond sensing unit is mounted to the conveyor system using a movablerobotic arm or actuator and is configured to capture at least one oftwo-dimensional or three-dimensional data at various aspects; whereinthe conveyor system is configured to manipulate the plants in order forthe second sensing unit to capture data from various angles. 15-18.(canceled)
 19. A crop monitoring system comprising: a mobile sensingunit including a sensor array and configured to navigate an arrangementof plants within a growing facility and generate raw data correspondingto the plants; a computing device comprising a processor, at least onememory module, an input device, and a communication module, wherein thecomputing device is in communication with the mobile sensing unit andconfigured to generate user information about environmental and plantconditions within the growing facility based on the raw data, a userinput, or both; and a graphical user interface in communication with thecomputing device and configured to display the user information aboutenvironmental and plant conditions within the growing facility andreceive user input to navigate the user information about environmentaland plant conditions within the growing facility.
 20. The cropmonitoring system of claim 19, wherein the graphical user interface islocated remotely from the mobile sensing unit.
 21. The crop monitoringsystem of claim 19, wherein the environmental and plant conditionswithin the growing facility presented via the graphical user interfaceinclude information relating to at least one of: temperature, planthealth status, plant stress status, disease, pests, plant performance,growth status, or yield predictions, and one or more of: wherein theuser can interact with the graphical user interface in order to displaythe environmental and plant conditions within the growing facility at alarge scale facility-level or a small scale bench-level; wherein thegraphical user interface is further configured to generate and present agrid-like map of the growing facility, including graphical indicationsof the environmental and plant conditions for individual zones withinthe growing facility; wherein the graphical user interface is furtherconfigured to generate and present information related to cultivationtask records and procedures. 22-24. (canceled)
 25. The crop monitoringsystem of claim 1, wherein the mobile sensing unit is further configuredto emit a light pulse to sense a plant's response to at least one ofvisible or invisible light pulses. 26-29. (canceled)
 30. Acomputer-implemented method for processing crop information, the methodimplementing an application processing system for use in generating andprocessing information related to environmental and plant conditionswithin a growing facility and displaying same, the method comprising:generating raw data via a mobile sensing unit, where the raw datacorresponds to one or more characteristics of one or more plants;receiving user input information in one of multiple available inputformats through an input interface; processing the raw data and userinput information to create a curated data set comprising processedimages representative of the crop information; comparing the curateddata set against a pre-existing database of domain data; determining,based at least in part on the comparison of the curated data set, thespecific environmental and plant conditions relative to the crop beingprocessed; generating a graphical user interface using a GUI generator;and displaying the information related to the environmental and plantconditions.
 31. The method of claim 30, wherein the graphical userinterface is interactive and the method further comprises manipulatingdisplayed information.
 32. The method of claim 30, wherein processingthe raw data further comprises performing a localization process toenhance the quality of the displayed information. 33-34. (canceled)